Mobile phone associated applications represent a rapidly growing market that offers users a number of helpful tools for a wide variety of tasks. Recent achievements in hardware and signal processing have increased the usability of these tools and give users opportunities to carry out accurate measurements on their own.
In parallel, there is growing demand for portable devices suitable for self-assessment of diet with an especially strong need for patients with Diabetes Mellitus of type I. The total number of persons with Diabetes is estimated to 400 million, and this number will substantially increase in the next few decades. One of the critical tasks for persons with diabetes is the control of the amount and type of food intake. For them, diet affects glycaemia much more than for healthy individuals. Clinical studies have shown that for children and adolescents on intensive insulin therapy an inaccuracy of ±10 g in carbs counting does not deteriorate the post-prandial control, while a ±20 g variation significantly impacts the postprandial glycaemia
Food intake estimation is a non-trivial task due to a wide variety of food types and complex irregular shapes of servings. Image processing and computer vision techniques have made some progress, but numerous uncertainties remain and cumulate (recognition of the type of food, estimation of the volume, diverse lighting conditions etc). Regarding the estimation of volume, which is a key parameter, available 3D scanning techniques for industrial activities—using medium power consumption lasers to scan objects and reconstruct 3D shapes—are not adapted to mobile and personal environments. Industrial lasers require important sources of energy, may be dangerous for the eyes and may require several image acquisition devices.
The patent literature is developed for 3D or volume estimation in industrial environments, but scarce for mass-market type environments.
EP2161537 discloses a position measuring apparatus including a first irradiating part that irradiates a first beam to an object, a second irradiating part that irradiates a second beam to the object, a capturing part that captures images of the object, a processing part that generates a first difference image and a second difference image by processing the images captured by the capturing part, an extracting part that extracts a contour and a feature point of the object from the first difference image, a calculating part that calculates three-dimensional coordinates of a reflection point located on the object based on the second difference image, and a determining part that determines a position of the object by matching the contour, the feature point, and the three-dimensional coordinates with respect to predetermined modelled data of the object. This system presents drawbacks.
Shang et al: “Dietary intake assessment using integrated sensors and software”, Proceedings of SPIE, Vol. 8304 page 830403, describes a system consisting of an mobile device that integrates a smartphone and an integrated laser package; software on the smartphone for data collection and laser control; an algorithm to process acquired data for food volume estimation, and a database and interface for data storage and management. The laser package creates a structured light pattern, in particular a laser grid.
The system collects videos with slow movement of the camera around the food, stabilized at several positions and collects video sequences. The laser is turned on and off during the video collection, resulting in video frames with and without laser grids alternatively. As the motion between two adjacent frames is considered small, the laser grid lines can be extracted by subtracting the non-grid images from the grid images.
Since the smartphone has only moderate computational power it is suitable for data collection but not for volume estimation. Therefore, acquired grid videos are transferred to a server for further processing. Furthermore, the food types are manually identified by the user, while the selection of several pairs of images whose motion is small is performed manually too. Those limit the system's usability.
There is a need for methods, systems and portable devices of estimating volume of objects (e.g. food) and to derive food nutritional values and insulin bolus advice thereof.